Database search system and method of determining a value of a keyword in a search

ABSTRACT

Methods of determining values of keywords in an internet search are described. According to one aspect of the invention, a method comprises steps of receiving keywords entered for a plurality of searches; detecting converted transactions associated with the plurality of searches; analyzing the converted transactions; and determining values associated with the keywords based upon the converted transactions. According to other aspects of the invention, methods for recommending subsets of keywords and for recommending keywords based upon converted transactions and click through rates are disclosed. A database search system is also disclosed.

FIELD OF THE INVENTION

The present invention relates generally to database searching, and inparticular, to a database search system and a method of determining avalue of a keyword or set of keywords in a database search.

BACKGROUND OF THE INVENTION

Generally, in a web-based search on an Internet search engine, a userenters a search term comprising one or more keywords, which the searchengine then uses to generate a search result list comprising of webpages that the user may access via a hyperlink. There are many ways inwhich a search engine can return the results list. There are searchengines that use automated search technology, which relies in large parton complex, mathematics-based database search algorithms that select andrank web pages based on multiple criteria such as keyword density andkeyword location. The search results generated by such mechanisms oftenrely on blind mathematical formulas and may be random and evenirrelevant. These search engines often catalog search results that relyon invisible web site descriptions, or “meta tags”, that are authored byweb site promoters. It is not uncommon for web site owners to tag theirsites as they choose in attempt to attract additional consumer attentionat little to no marginal cost and no guarantee that the website'scontent is actually relevant to the meta tags used on its site.

Internet advertising can offer a level of targetability, interactivity,and measurability not generally available in other media. With theproper tools, Internet advertisers have the ability to direct and targettheir messages to specific groups of consumers and receive promptfeedback as to the effectiveness of their advertising campaigns such asthe services offered by commercial search engine providers such asOverture (http://www.overture.com). Many of the traditional paradigms ofadvertising and search engine algorithms fail to offer appropriateoptions to businesses or advertisers or to maximize the delivery ofrelevant information via the web to interested parties in acost-effective manner for those businesses and advertisers. Ideally, website promoters or advertisers should be able to control their placementin search result listings so that their listings appear in searches thatare relevant to the content of their web site and to control their modeof participation and any associated costs so that such listings aredesirable or effective for their marketing objectives in using acommercial search engine. Search engine functionality needs tofacilitate an online marketplace which offers consumers quick, easy andrelevant search results while providing Internet advertisers andpromoters with a cost-effective way to target consumers in a manner theydetermine most appropriate for their business goals. In this onlinemarketplace, companies selling products, services, or information willbe able to determine the options that best suit its advertising and costneeds and appear in desirable positions on a search result listgenerated by an Internet search engine.

Since advertisers generally must pay for each referral generated throughthe search result lists generated by the search engine, advertisers havean incentive to select and bid on those search keywords that are mostrelevant to their web site offerings and are most likely to achievetheir marketing goals, e.g. inducing searchers to purchase theadvertisers products or services.

A referral can be of any one of a number of types. One type of referralis an impression referral, whereby the advertiser's listing appears inthe search results list. Another type of referral is a click-throughreferral, whereby a consumer clicks on the advertiser's listing. Stillanother type of referral is an action referral, whereby after theconsumer has clicked on the advertiser's listing, the consumer takesfurther action in connection with the advertiser's web site. Thesefurther actions include, for example, actions such as registering withthe advertiser's site, participating in a promotion in connection withthe advertiser's site, and/or purchasing a good or service from theadvertiser. Advertisers can bid on one or more of the various types ofreferrals. In the case where the advertiser bids on more than one typeof referral, a bid will comprise multiple elements, one element per typeof referral bid upon. The higher the market value of an advertiser'sbid, the higher the advertiser's position on a search result list. Thehigher an advertiser's position on a search result list, the higher thelikelihood of a referral; that is, the higher the likelihood that aconsumer will be referred to the advertiser's web site through thesearch result list by clicking on the result in that position on thepage.

The advertiser influences a position for a search listing in theadvertiser's account by first selecting a search term relevant to thecontent of the web site or other information source to be listed. Theadvertiser enters the search term and the description into a searchlisting. The advertiser influences the position for a search listingthrough a continuous online competitive bidding process. The biddingprocess occurs when the advertiser enters a new bid amount, which ispreferably a money amount, for a search listing. The commercial searchengine then compares this bid amount with all other bid amounts for thesame search term, and generates a rank value for all search listingshaving that search term. The rank value generated by the bidding processdetermines where the advertiser's listing will appear on the searchresults list page that is generated in response to a query of the searchterm by a searcher or user on the computer network. A higher bid by anadvertiser will usually result in a higher rank value and a moreadvantageous placement although other factors besides bids may also beconsidered in ranking the search results.

Thus, when a user performs a search on such a search engine, the resultsare conventionally sorted based on how much each advertiser has bid onthe user's search term. Because different users will use different wordsto find the same information, it is important for an advertiser to bidon a wide variety of search terms in order to maximize the traffic tohis site. The better and more extensive an advertiser's list of searchterms, the more traffic the advertiser can generate for their website.In cases where the advertiser is able to track relative keywordperformance for directing traffic to their website.

However, selecting search terms by an advertiser can be a challenge.Good search terms have three significant properties: they are relevantto the advertiser's site content, they are popular enough that manyusers are likely to search on them, and they provide good value in termsof the expected return from the traffic they send to the advertiser'swebsite. An advertiser willing to take the time to consider all thesefactors may get good results. Selecting successful bidding strategiesusing conventional tools can also be unpredictable because suchconventional tools do not take converted transactions into account andcannot aggregate price change data over time. Not only do values ofsearch keywords change over time, conventional tools for helping selecta bidding strategy fail to make recommendations for a search based uponconverted transactions or other individual advertiser objectivesinfluenced by time.

Accordingly, there is a need for an improved database search system andmethod of determining a value of a keyword in a search over time.

SUMMARY OF THE INVENTION

Methods of determining values of keywords in an internet search aredescribed. According to one aspect of the invention, a method comprisessteps of receiving keywords entered for a plurality of searches;detecting converted transactions associated with the plurality ofsearches; analyzing the converted transactions; and determining valuesassociated with the keywords based upon the converted transactions.According to other aspects of the invention, methods for recommendingsubsets of keywords and for recommending keywords based upon convertedtransactions and click through rates are disclosed.

A database search system is also disclosed. The database search systempreferably comprises a storage element storing information related toconverted transactions for the purchase of goods online by way of thedata search system; a processing system coupled to the storage element,the processing system generating recommendations for selecting keywordsfor advertisers associated with the database based upon the convertedtransactions; and a graphical user interface coupled to the processingsystem, the user interface enabling advertisers to access therecommendations for selecting keywords. According to other aspects ofthe invention, advertiser web servers and client devices are alsodisclosed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system for enabling a database search according to thepresent invention;

FIG. 2 is a block diagram of a system for enabling advertisers to managean account associated with database searching according to the presentinvention;

FIG. 3 is a timing diagram showing the steps in generating a keywordrecommendation according to the present invention;

FIG. 4 is a table showing a probability of a sale and an average dollarper order for a given keyword according to the present invention;

FIG. 5 is a table showing normalized values of probabilities forkeywords according to the present invention;

FIG. 6 is a table showing an entropy value for keyword sets according tothe present invention;

FIG. 7 is a flowchart showing a method of recommending search termsbased upon converted transactions according to the present invention;

FIG. 8 is a flowchart showing a method of recommending subsets ofkeywords according to the present invention; and

FIG. 9 is a flowchart showing a method of recommending search termsbased upon transaction completion probabilities and click through ratesaccording to the present invention.

DETAILED DESCRIPTION OF THE DRAWINGS

Referring now to the drawings, FIG. 1 is an example of a distributedsystem 10 configured as client/server architecture used in a preferredembodiment of the present invention. A “client” is a member of a classor group that uses the services of another class or group to which it isnot related. In the context of a computer network, such as the Internet,a client is a process (i.e. roughly a program or task) that requests aservice which is provided by another process, known as a server program.The client process uses the requested service without having to know anyworking details about the other server program or the server itself. Innetworked systems, a client process usually runs on a computer thataccesses shared network resources provided by another computer running acorresponding server process. However, it should also be noted that itis possible for the client process and the server process to run on thesame computer.

A “server” is typically a remote computer system that is accessible overa communications medium such as the Internet. The client process may beactive in a second computer system, and communicate with the serverprocess over a communications medium that allows multiple clients totake advantage of the information-gathering capabilities of the server.Thus, the server essentially acts as an information provider for acomputer network.

The block diagram of FIG. 1 therefore shows a distributed system 10comprising a plurality of client computers 12, a plurality of advertiserweb servers 14, an account management server 22, and a search engine webserver 24, all of which are connected to a network 20. The network 20will be hereinafter generally referred to as the Internet. Although thesystem and method of the present invention is specifically useful forthe Internet, it should be understood that the client computers 12,advertiser web servers 14, account management server 22, and searchengine web server 24 may be connected together through one of a numberof different types of networks. Such networks may include local areanetworks (LANs), other wide area networks (WANs), and regional networksaccessed over telephone lines, such as commercial information services.The client and server processes may even comprise different programsexecuting simultaneously on a single computer.

The client computers 12 can be conventional personal computers (PCs),workstations, or computer systems of any other size. Each client 12typically includes one or more processors, memories, input/outputdevices, and a network interface, such as a conventional modem. Theadvertiser web servers 14, account management server 22, and the searchengine web server 24 can be similarly configured. However, advertiserweb servers 14, account management server 22, and search engine webserver 24 may each include many computers connected by a separateprivate network. In fact, the network 20 may include hundreds ofthousands of individual networks of computers.

The client computers 12 can execute web browser programs 16, such as aconventional browser program well known in the art, to locate the webpages or records 30 stored on advertiser server 14. The browser programs16 allow the users to enter addresses of specific web pages 30 to beretrieved. These addresses are referred to as Uniform Resource Locators,or URLs. In addition, once a page has been retrieved, the browserprograms 16 can provide access to other pages or records when the user“clicks” on hyperlinks to other web pages. Such hyperlinks are locatedwithin the web pages 30 and provide an automated way for the user toenter the URL of another page and to retrieve that page. The pages canbe data records including as content plain textual information, or morecomplex digitally encoded multimedia content, such as software programs,graphics, audio signals, videos, and so forth.

In a preferred embodiment of the present invention, shown in FIG. 1,client computers 12 communicate through the network 20 with variousnetwork information providers, including account management server 22,search engine server 24, and advertiser servers 14 using thefunctionality provided by a HyperText Transfer Protocol (HTTP), althoughother communications protocols, such as FTP, SNMP, TELNET, and a numberof other protocols known in the art, may be used. Preferably, searchengine server 24, account management server 22, and advertiser servers14 are located on the World Wide Web.

As discussed above, at least two types of server are contemplated in apreferred embodiment of the present invention. The first servercontemplated is an account management server 22 comprising a computerstorage medium 32 and a processing system 34. A database 38 is stored onthe storage medium 32 of the account management server 22. The database38 contains advertiser account information. It will be appreciated fromthe description below that the system and method of the presentinvention may be implemented in software that is stored as executableinstructions on a computer storage medium, such as memories or massstorage devices, on the account management server 22. Conventionalbrowser programs 16, running on client computers 12, may be used toaccess advertiser account information stored on account managementserver 22. Preferably, access to the account management server 22 isaccomplished through a firewall, not shown, which protects the accountmanagement and search result placement programs and the accountinformation from external tampering. Additional security may be providedvia enhancements to the standard communications protocols such as SecureHTTP or the Secure Sockets Layer.

The second server type contemplated is a search engine web server 24. Asearch engine program permits network users, upon navigating to thesearch engine web server URL or sites on other web servers capable ofsubmitting queries to the search engine web server 24 through theirbrowser program 16, to type keyword queries to identify pages ofinterest among the millions of pages available on the World Wide Web. Ina preferred embodiment of the present invention, the search engine webserver 24 generates a search result list that includes, at least inpart, relevant entries obtained from and formatted by the results of thebidding process conducted by the account management server 22. Thesearch engine web server 24 generates a list of hypertext links todocuments that contain information relevant to search terms entered bythe user at the client computer 12. The search engine web servertransmits this list, in the form of a web page, to the network user,where it is displayed on the browser 16 running on the client computer12. A presently preferred embodiment of the search engine web server maybe found by navigating to the web page at URL http://www.goto.com/. Inaddition, the search result list web page, an example of which ispresented in FIG. 7, will be discussed below in further detail.

Search engine web server 24 is connected to the Internet 20. In apreferred embodiment of the present invention, search engine web server24 includes a search database 40 comprised of search listing recordsused to generate search results in response to user queries. Inaddition, search engine web server 24 may also be connected to theaccount management server 22. Account management server 22 may also beconnected to the Internet. The search engine web server 24 and theaccount management server 22 of the present invention address thedifferent information needs of the users located at client computers 12.

For example, one class of users located at client computers 12 may benetwork information providers such as advertising web site promoters orowners having advertiser web pages 30 located on advertiser web servers14. These advertising web site promoters, or advertisers, may wish toaccess account information residing in storage 32 on account managementserver 22. An advertising web site promoter may, through the accountresiding on the account management server 22, participate in acompetitive bidding process with other advertisers. An advertiser maybid on any number of search terms relevant to the content of theadvertiser's web site. In one embodiment of the present invention, therelevance of a bidded search term to an advertiser's web site isdetermined through a manual editorial process prior to insertion of thesearch listing containing the search term and advertiser web site URLinto the database 40. In an alternate embodiment of the presentinvention, the relevance of a bidded search term in a search listing tothe corresponding web site may be evaluated using a computer programexecuting at processor 34 of account management server 22, where thecomputer program will evaluate the search term and corresponding website according to a set of predefined editorial rules.

The higher bids receive more advantageous placement on the search resultlist page generated by the search engine 24 when a search using thesearch term bid on by the advertiser is executed. In a preferredembodiment of the present invention, the amount bid by an advertisercomprises a money amount that is deducted from the account of theadvertiser for each time the advertiser's web site is accessed via ahyperlink on the search result list page. A searcher “clicks” on thehyperlink with a computer input device to initiate a retrieval requestto retrieve the information associated with the advertiser's hyperlink.Preferably, each access or “click” on a search result list hyperlinkwill be redirected to the search engine web server 24 to associate the“click” with the account identifier for an advertiser. This redirectaction, which is not apparent to the searcher, will access accountidentification information coded into the search result page beforeaccessing the advertiser's URL using the search result list hyperlinkclicked on by the searcher. The account identification information isrecorded in the advertiser's account along with information from theretrieval request as a retrieval request event. Since the informationobtained through this mechanism conclusively matches an accountidentifier with a URL, accurate account debit records will bemaintained. Most preferably, the advertiser's web site description andhyperlink on the search result list page is accompanied by an indicationthat the advertiser's listing is a paid listing. Most preferably, eachpaid listing displays a “cost to advertiser,” which is an amountcorresponding to a “price-per-click” paid by the advertiser for eachreferral to the advertiser's site through the search result list.

A second class of users at client computers 12 may comprise searchersseeking specific information on the web. The searchers may access,through their browsers 16, a search engine web page 36 residing on webserver 24. The search engine web page 36 includes a query box in which asearcher may type a search term comprising one or more keywords.Alternatively, the searcher may query the search engine web server 24through a query box hyperlinked to the search engine web server 24 andlocated on a web page stored at a remote web server. When the searcherhas finished entering the search term, the searcher may transmit thequery to the search engine web server 24 by clicking on a providedhyperlink. The search engine web server 24 will then generate a searchresult list page and transmit this page to the searcher at the clientcomputer 12.

The searcher may click on the hypertext links associated with eachlisting on the search results page to access the corresponding webpages. The hypertext links may access web pages anywhere on theInternet, and include paid listings to advertiser web pages 18 locatedon advertiser web servers 14. In a preferred embodiment of the presentinvention, the search result list also includes non-paid listings thatare not placed as a result of advertiser bids and are generated by aconventional World Wide Web search engine. The non-paid hypertext linksmay also include links manually indexed into the database 40 by aneditorial team. Most preferably, the non-paid listings follow the paidadvertiser listings on the search results page.

Turning now to FIG. 2, a diagram shows menus, display screens, and inputscreens presented to an advertiser accessing the account managementserver 22 through a conventional browser program 16. The advertiser,upon entering the URL of the account management server 22 into thebrowser program 16 of FIG. 1, invokes a login application, discussedbelow as shown at screen 110 of FIG. 2, running on the processing system34 of the server 22. Once the advertiser is logged-in, the processingsystem 34 provides a menu 120 on a graphical user interface that has anumber of options and further services for advertisers. These items,which will be discussed in more detail below, cause routines to beinvoked to either implement the advertiser's request or request furtherinformation prior to implementing the advertiser's request. In oneembodiment of the present invention, the advertiser may access severaloptions through menu 120, including requesting customer service 130,viewing advertiser policies 140, performing account administration tasks150, adding money to the advertiser's account 160, managing theaccount's advertising presence on the search engine 170, and viewingactivity reports 180. Context-specific help 190 may also generally beavailable at menu 120 and all of the above-mentioned options. Finally,keyword recommendations 195 also preferably provide beneficialinformation to advertisers. As will be described in more detail inreference to the remaining figures, the systems and methods of thepresent invention determine values associated with keywords, and rankthe keywords based on transaction completion probability. Such a rankingenables maximizing transaction values over a subset of keywords. Thesystems and methods also preferably calculate average dollar values perorder and generate return on investment values.

Turning now to FIG. 3, a timing diagram shows the steps in generating akeyword recommendation according to the present invention. An importantgoal of sponsored search is to get key placements on search/paid listingsites, and drive traffic to those sponsored sites. According to oneaspect of the invention, an end-to-end solution not only drives trafficto websites, but also provides a higher degree of confidence foradvertisers that such traffic to websites will lead to convertedtransactions. A converted transaction could include an actual purchaseby a user, or could also include any event completion, such as insertionof item into a shopping cart, or linking off to an (internal orexternal) site. As will be set forth in more detail below, probabilisticmodels for estimating and determining how various keywords translate toorder placement dollar amounts is described. An important aspect of thisanalysis is the approach of utilizing end-to-end information fordetermining associations of search keywords to completed e-commercetransactions, and the ability to recommend which sponsored keywords canbe favorable from a transaction perspective.

Such recommendations for keywords not only maximizes advertisingrevenue, but also potentially increases revenue generated by a searchengine provider through the sales of goods by providing end-to-endsolutions to customers. In particular, a user searching the internetusing a search engine based upon some keywords will get a list of insidelistings related to the provider of the search engine, paid searchresults, and normal search results. With millions of users receivingsearch results by accessing a search engine, there is a tremendousopportunity to funnel these users to other areas controlled by theprovider of the search engine. Perhaps more importantly, these possibly“window-information-shopping” customers can be converted into paidcustomers by identifying and driving them to compellingproducts/services that are of interest and value to them.

Small Business services are one such compelling service area thatprovide a variety of substantial services for small business users andthe customers of these small business users. FIG. 3 illustrates theend-to-end approach of a search as a funnel into the network for smallbusiness e-commerce transactions. Keywords entered on an internet search302 get potential customers into the network by generating searchresults 304. These search results can lead a user to online merchants306 affiliated with the provider of the search engine. Customers make atransaction which translates to revenue for a merchant as well as theprovider of the search engine. Data analysis 310 associated with thetransaction is then generated to provide estimates of the value for eachkeyword, as will be described in more detail below. Finally, a keywordrecommendation 312 can be generated by the provider of the search engineto an advertiser to allow the advertiser to select the best keywords fora given budget.

According to one aspect of the invention, the translation of keywordsthat actually generate e-commerce transactions, rather than just basedon many searches that lead to a site/store affiliated with the providerof the search engine, provides more relevant information. Thus, the toolof the present invention is a more robust end-to-end model forrecommending keywords for sponsored search listings. As will bedescribed in more detail below, the keyword recommendations arepreferably periodically or continuously run to provide current orupdated values associated with keywords. Of course, this iterativeprocess can be boostrapped with normal search keywords that sendcustomers to the sites affiliated with the provider of the searchengine. This information can be integrated with other properties such asinfluencing ranking of the site in shopping listing.

Turning now to FIG. 4, a table shows a probability of a sale and anaverage dollar per order for a given keyword according to the presentinvention. In order to generate a keyword recommendation based uponconverted transactions, the probability that some transaction iscompleted on a particular store or site “S” when user searches withkeyword “k” must be determined. The probability could be defined to be:P(T|k ,S). This probability can be estimated using Bayes' rule:${P\left( {{T❘k},S} \right)} = \frac{{P\left( {k,{S❘T}} \right)}*\left( {P(T)} \right.}{P\left( {k,S} \right)}$

The conditional probability P(k,S|T) can be computed using thee-commerce transaction completion data reflecting the occurrences ofkeyword “k” with purchases of any items in store site “S”. Here the P(T)can be estimated using the general transaction probability. P(k,S) isestimated to be the unigram keyword probability. An Expected TransactionValue (i.e. average dollar value per order) can be computed as follows:${E\left( {{T❘k},S} \right)} = {{P\left( {{T❘k},S} \right)}*\frac{\sum\limits_{\forall t}{R\left( {{t❘k},S} \right)}}{C\left( {{t❘k},S} \right)}}$

Here R(t) is the transaction value of order “t”. As shown in FIG. 4, anexample of the probability of a converted transaction and an averagedollar per order for particular keywords is shown. A few of the topprobabilistic keywords are listed in the table above. It should be notedthat there is a variance in the average dollar amount per order. Thisinformation can be further used to compute the ROI based ranking of thesearch keywords. Expected number of transactions for a given keywordtimes the expected transaction value for a given keyword will estimatethe ROI for listing that keyword.

Turning now to FIG. 5, a table shows normalized values of probabilitiesfor keywords according to the present invention. In particular, thetable below shows the normalized ROI probability (-Logarithm), and liststhe top 10 keywords. It should be noted that the high dollar orders moveup in positions over the lower revenue generating orders. Also, somesimplifying assumptions, such as an assumption that a return on an orderis proportional to the cost of the order, can be been made.

Turning now to FIG. 6, a table shows an entropy value for keyword setsaccording to the present invention. Based on the entropy measure (i.e. avalue of keyword or combination of keywords for a given budget),keywords can be picked with the highest entropy depending on theplacement slot cost for each keyword. The keywords can be ranked basedon the conditional transaction completion probability to maximize returnon investment (ROI). In order to select a subset of keywords for maximumROI, certain constraints can be applied. For example, the total cost ofkeywords is less than the maximum constrained budget, and the return forgiven keywords is maximized (i.e. Maximize(E(T|K,S))) over subset ofkeywords K. At each step of the iteration, different set lengths can bechosen. Furthermore, at each replacement step, the keyword replacementis retained if the transaction return value is greater than without thekeyword replacement.

Many methods of computing subsets of keywords that maximize ROI can beused. In the algorithm listed below, we compute the Expected return on atransaction based on a keyword, or a set of keywords, and iterate oversubsets and retain the subsets with a maximum entropy value.

Entropy of a set of keywords {t₁,t₂,t₃ . . . } is defined as${{Entropy}\left( {{t1},{t2},{{t3}\quad\ldots}} \right)} = {- {\sum\limits_{\forall t}{{E\left( {{t❘k},S} \right)}*{\log\left( {E\left( {{t❘k},S} \right)} \right.}}}}$

-   -   1. Define the maximum keyword list length, entropy threshold,        and maximum recommendation list length.    -   2. Pick top ROI keywords for computing recommendation word sets.    -   3. Iterate over the number of keywords to be chosen    -   4. For each subset with this cardinality, compute the total cost        of sponsored listing of that keyword subset (sum of individual        keyword sponsored listing costs for all keywords in that        subset). If less than the available budget goto next step, else        iterate over next subset of keywords.    -   5. For this subset of keywords compute the entropy of that        keyword set.    -   6. If the computed entropy is more than the entropy of any        existing set in the recommendation list, then insert into the        list.    -   7. Iterate over next keyword subset.    -   8. The final list of keyword lists on the recommendation list is        the desired output.

Referring to the example in the table of FIG. 6, assuming that“Storebrand 1” and “Online Vitamins” have placement costs of say $100and the rest have placement costs of say $50. Applying the maximumentropy algorithm, we have the following ordering of keyword sets, basedon a budget constraint of $100. It should be noted that beyond“Storebrand 1” in this example, other keyword sets (with the pricingconstraint) had higher entropy for other keyword pairs. Furthermore, thekeywords “Storebrand 1” and “Online Vitamins” are also already listed inthe paid sponsor listings, therefore providing a very dynamic andintegrated mechanism.

Turning now to FIG. 7, a flowchart shows a method of recommending searchterms based upon converted transactions according to the presentinvention. In particular, keywords entered for a plurality of searchterms are received at a step 702. Converted transactions associated withthe plurality of searches are then detected at a step 704. The convertedtransactions are then analyzed at a step 706. Values associated with thekeywords based upon a plurality of searches are determined at a step708. A keyword transaction probability is then estimated at a step 710,and expected transaction values are generated at a step 712.Recommendations for search terms are then made based upon productspurchased at a step 714. Finally, values associated with keywords areperiodically updated at a step 716.

Turning now to FIG. 8, a flowchart shows a method of recommendingsubsets of keywords according to the present invention. In particular,keywords entered for a plurality of search terms are received at a step802. Converted transactions associated with the plurality of searchesare then detected at a step 804. Values associated with the keywordsbased upon a plurality of searches are determined at a step 806. Anaverage dollar per order is then calculated at step 808. The keywordsare then ranked based upon conditional transaction completionprobabilities at a step 810. A return on investment ranking of keywordsis generated at a step 812. An expected transaction values over a subsetof keywords is maximized at a step 814. Finally, the values associatedwith the keywords are periodically updated at a step 816.

Turning now to FIG. 9, a flowchart shows a method of recommending searchterms based upon transaction completion probabilities and click throughrates according to the present invention. In particular, keywordsentered for a plurality of search terms are received a step 902. Clickthrough rates of websites from the search results are determined at astep 904. Converted transactions associated with the plurality ofsearches are detected at a step 906. Values associated with the keywordsbased upon a plurality of searches are determined at a step 908.Keywords are then ranked based upon transaction completion probabilityand click through rates at a step 910. An average dollar per order isthen calculated at a step 912. A return on investment ranking ofkeywords is generated at a step 914. Expected transaction values aremaximized over a subset of keywords at a step 916. Finally, the valuesassociated with the keywords are periodically updated at a step 918.

In this summary, an end-to-end approach to sponsored searchrecommendations is described. Accordingly, the system and method of thepresent invention enables the collection of keyword data based ontransaction completion data; an estimation of conditional probabilitiesof each keyword translating to an e-commerce transactions; thecomputation of an expected transaction value generated by each keyword;and the integrate the above metrics with the overall search keywordprobabilities. Given a sponsored search budget, and bids on eachkeyword, the system of the present invention can compute the subset ofkeywords based on the probabilistic model generated for individualkeywords. Finally, the keyword recommendations can be iterated andimproved using a feedback adaptation process.

By utilizing search keywords that have resulted in completedtransactions, as opposed to searches that did not culminate intotransactions, an advertiser is provided with better information to bidon keywords. Furthermore, this order completion information is used tomodel the probabilities of completed transactions for differentkeywords, and also expected returns on the keywords. The systems andmethods of the present invention also open up many productizationopportunities, such as sponsored keyword recommendation based onintegrated information across a network. For example, service can beprovided to small business merchants as a tool with a monthly fee.

It can therefore be appreciated that the new and novel database searchsystem and method of determining a value of a keyword in a search hasbeen described. It will be appreciated by those skilled in the art that,particular the teaching herein, numerous alternatives and equivalentswill be seen to exist which incorporate the disclosed invention. As aresult, the invention is not to be limited by the foregoing embodiments,but only by the following claims.

1. A method of determining values of keywords in an internet search,said method comprising the steps of: receiving keywords entered for aplurality of searches returning a plurality of search results inresponse thereto; detecting click throughs by users on one or more ofsaid search results detecting converted transactions associated withsaid plurality of click throughs; analyzing said converted transactions;and determining values associated with said keywords based upon saidconverted transactions.
 2. The method of claim 1 wherein said step ofanalyzing said converted transactions comprises a step of periodicallyupdating values associated with keywords.
 3. The method of claim 1wherein said step of determining values associated with said keywordsbased upon said converted transactions comprises a step of generatingexpected transaction values.
 4. The method of claim 1 further comprisinga step of estimating a transaction probability for at least one keyword.5. The method of claim 4 further comprising a step of makingrecommendations for search terms based on said converted transactions.6. A method of determining values of keywords in an internet search,said method comprising the steps of: receiving keywords entered for aplurality of searches returning a plurality of search results; returningat least one search result listing; detecting converted transactionsassociated with said plurality of searches; ranking keywords based upona transaction probability; and providing a recommendation of a subset ofkeywords to optimize marketing objectives
 7. The method of claim 6further comprising a step of determining values associated withindividual keywords based upon a plurality of searches.
 8. The method ofclaim 6 further comprising a step of a determining a value associatedwith a set of keywords based upon a plurality of searches.
 9. The methodof claim 6 further comprising a step of calculating an average value perorder for a given keyword.
 10. The method of claim 6 further comprisinga step of generating a ranking of keywords based on theirreturn-on-investment value.
 11. The method of claim 10 furthercomprising a step of generating a relative ranking of keywords based ontheir relative return-on-investment value to advertisers.
 12. A methodof determining values of keywords in an internet search, said methodcomprising the steps of: receiving keywords entered for a plurality ofsearches; returning a plurality of search results; detecting clickthroughs by users on one or more said search results; determining clickthrough rates for websites from said plurality of searches; detectingconverted transactions associated with said plurality of searches; andranking keywords based upon transaction completion probability and clickthrough rates.
 13. The method of claim 11 further comprising a step ofdetermining values associated with said keywords based upon a pluralityof searches.
 14. The method of claim 11 further comprising a step ofmaximizing expected transaction values over a subset of keywords. 15.The method of claim 11 further comprising a step of calculating anaverage dollar per order for a given keyword.
 16. The method of claim 11further comprising a step of generating a return on investment rankingof keywords.
 17. A database search system comprising: a storage elementstoring information related to converted transactions for the purchaseof goods online by way of said data search system; a processing systemcoupled to said storage element, said processing system generatingrecommendations for selecting keywords for advertisers associated withsaid database based upon said converted transactions; and a userinterface coupled to said processing system, said user interfaceenabling advertisers to access said recommendations for selectingkeywords.
 18. The database search system of claim 16 wherein saidprocessing system further generates keyword transaction probabilitiesfor a plurality of keywords.
 19. The database search system of claim 16wherein said processing system further generates an average dollar perorder value for a given keyword.
 20. The database search system of claim16 wherein said processing system further generates a return oninvestment value for a given keyword.
 21. The database search system ofclaim 16 wherein said processing system further generates a transactionprobability for a subset of keywords.
 22. A database search systemhaving an account management server, said system comprising: a storageelement storing information related to converted transactions for thepurchase of goods online by way of said database search system; aprocessing system coupled to said storage element, said processingsystem generating recommendations for selecting keywords for advertisersassociated with said database based upon said converted transactions;and a graphical user interface enabling a plurality of web advertiser toaccess said recommendations for selecting keywords.
 23. The databasesearch system of claim 21 further comprising a search engine web servercoupled to said account management server.
 24. The database searchsystem of claim 22 further comprising a plurality of client deviceshaving access to said search engine web server, each client devicehaving a browser for accessing said search engine web server.
 25. Thedatabase search system of claim 23 further comprising an advertiser webserver coupled to said account management server.
 26. The databasesearch system of claim 24 wherein said advertiser web server receivesinformation from said account management server comprising at least onepiece of information from the group consisting of: transactioncompletion probability; an average dollar per order for a given keyword;a return on investment value for a given keyword; or a transactionprobability for a subset of keywords.